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ABSTRACT 

This final evaluation report describes the 
Curriculum-Based Vo:iational Assessment (CBVA) project of the 
Depaicment of Defense Dependents schools (DoDDS) . The program serves 
mildly handicapped (learning disabled and mildly retarded) secondary 
level students in 18 regular vocational education courses delivered 
in 45 middle, junior high, and senior high schools in DoDDS schools 
in Germany. CBVA assessment forms for 1,811 students were completed 
by 317 teachers, job placement coordinators, or work site 
supervisors. The project found that special education students were 
rated in the upper quartile for all 21 work-related behaviors 
indicating positive social adaptability jn this setting. Performance 
on generalized employability skills was mixed with students average 
or better in applying academic skills to vocational instruction but 
was less proficient in categories related to employer-employee 
relationships. Vocational teachers indicated that extra assistance 
was routinely provided, with proficiency levels dramatically improved 
by such assistance. Results are reported both textually and 
graphically with findings separated by sex and by grades 6-8 and 
grades 9-12 for the following categories: grouped work related 
behaviors; grouped generalized employability outcomes; occupational 
skills outcomes; interest/aptitude; individual work related 
behaviors; and individual generalized employability outcomes. 
Additional charts present results on teacher assistance for the same 
categories. (DB) 
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FOREWORD 



The following document represents the final program evaluation of the DoDDS-G 
Curriculum-Based Vocational Assessment (CBVA) project. The evaluation was 
conducted by Dr. Mahlon E. Porter, Branch Chief, Education Division, DoDDS- 
Germany. 

The many personnel vrfio helped develop the CBVA Model and especially the 
secondaiy le^/el vocational teachers and Cooperative Vfork E>qDerience 
Coordinators vho participated in rating ^)ecial needs students in their 
classes are canmended for tl^ieir efforts. The project is among the most 
cotprehensive effort to date by a United States school system to design 
and implement a CBVA procedure. The aoooiTpanying research is also contri- 
buting to the needs of the profession at large throu^ the rare documentation 
of how well learning iirpaired students achieve in regular vocational programs. 

The CBVA Model as well as the research has reached United States and 
international audiences throu^ publication of jouma]. articles and 
presentations at conferences. The authors of a recent text on CETn/A have 
recognized DoDDS-Germany for its contribution to many of the ideas 
ejqjressed and documents used in the text. 

DoH}S-Germany Educators can take pride in their pioneering achievements 
with regard to CBVA development. Many school systems are interested in 
the procedural manual and are developing similar approaches thanks to 
the professionalism demonstrated in this project. 
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HOW VOCATiaiAL TEACHERS RATE OASSROCT l 



PERFORMANCE OF STUDENIS WITO MILD 
HANDICAPS USING CURRICUIDM-BASED VDCATICmL 



ASSESSMEOT PROCEDURES 



A Curriculum-Based Vocational Assessmsnt (CBVA) nodel was develcped, field 
tested, inodifisd, and iinplemented in 45 middle, junior hi^, and senior 
high schools in the Department of Defense Dependents Schools (DoDDS) in 
Germany during the period of August 1984 to June 1987 (Porter & Stodden, 
1986) . The develcpnental plan was structured around six key steps to CBVA, 
(lanacone & Laconte, 1986) vtiich provided for: (1) identification of key 
personnel, (2) conducting a ocuprehensive search for program models, etc., 
(3) establishing basic considerations for the model, (4) establishing an 
operational plan to implement the process, (5) piloting and evaluating CBVA 
activities, and (6) implementing and evaluating the systemwide program. 
The activities culminated in a CBVA model description, a handbook of 
procedures, preliminary vocational assessment rating forms, and a plan for 
evaluating the project (Stodden, lanacone, 1985) . 

The formative evaluation (Stodden, 1986) following the field test provided 
important feedback fron 175 vocational teachers and 23 other special educa- 
tion teachers and counselors. The results demonstrated high confidence in 
the reliability and validity status of the CBVA rating forms and proffered 
many suggestions regarding procedural modifications subsequently made in 
the revised CBVA handbook (Stodden & lanacone, 1986) . A prelimLnary system 
analysis of student proficiency ratings, as assessed by their vocational 
teachers in regular vocational courses, revealed strong evidence of voca- 
tional teacher support and overall positive student achievement in the 
mainstream (Porter, 1987) . 

The current evaluation collects and analyzes demographic data concerning 
student proficiency levels following full scale iitplementation of the CBVA 
model during School Year 1986-87. 

PURPOSE 

TSie CBVA project is designed to assess the performance of mildly handi- 
caK)ed secondary level students after oonpletion of regular vocational 
courses. Curriculum content outocmes for each course includes work 
related behaviors, generalized employability outcanes, and occupational 
skill outcomes. Ihe primary intent for the use of CBVA data is to predict 
individual student conpetence throu^ analysis of single and multiple 
ratings over extended school experiences in vocational subjects. The 
purpose of the program implementation evaluation is to discover the level 
of grxx?) performance in specific vocational courses and across the spectrum 
of vocational offerings-. Answers to the following questions are sou^t: 
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1. HcM well do students perform on work related behaviors, generalized 
eirplqyability outcomes, cirri ocxopational skill cuccomes in each 
vocatioral cxxirse offering? 

2. What levels of interest and aptitude do teachers rate students as 
having for the content areas after instruction? 

3. HcM proficient are students in specific work related behaviors 
axxi generalized enployability outocmes v*ien rated across the 
spectrum of vocational courses? 

4 . How inuch e>ctra assistance do students require from vocational 
teachers in becOTiing proficient in work related iDehavicrs, 
enployability outcomes, and occipational skill outcomes? 

5. What are the trends in proficiency ratings, interests, and 
aptitudes by gender? 

6. Can student ratings received from vocational teachers be used to 
predict ejq^ected work related behaviors and generalized enploya- 
bility outcomes in subsequent work environirents? 

The evaluation is limited to the demograjdiic c±>servations of student 
proficiency levels and vocational teacher rating trends, neither of vtiich 
insures that the CBVA Model contributes to student achievement nor teacher 
performance • The data does pmvide a oonprehensive view of the achievement 
levels of mildly handicapped students mains-. reamed in vocational courses 
from 6th to 12th grades and of the feasibility of collecting curriculum- 
based performance data from vocational teachers. Both outcomes are 
considered iirportant to the growing desire within the profession to apply 
CBVA techniques to normal vocational instructional environments (Cobb, 1983) 
(Col±) fit Larkin, 1985) . 

MEIHOD 

CBVA rating forms (See Page 3) were distributed to vocatioml teachers and 
Coc^rative Work E)q)erienoe (Offi) job sipervisors. Instructors were asked 
to assess all mildly handicapped students with Individualized Education 
Programs v*io were enrolled in vocational courses at the end of both first and 
second semesters. At middle/ junior hi^ school levels, exploratory courses 
could terminate as early as 6 weeks and be included in the scudy. The data 
were groiped for analysis according to vAiether stojdents ware enrolled Ln grades 
6 to 8 or in grades 9 to 12. The gmi^DS were composed of 23 hi^ schools with 
979 ratings provided for students in grades 9 to 12 and 36 middle/ junior high 
schools with 832 ratings prrvided for students in grades 6 to 8. Personnel 
oonpleting CBVA forms included 120 vocational teachers frran middle/junior high 
school levels and 152 teachers from senior hic^ school levels. Ihere were 
149 hi^ sdhool CWF students assessed by 45 CWE jc* placement coordinators or 
their work site supervisors. 

Content validity was established during the 1985-86 school year formative 
evaluation and further ijnproved throu^ modifications made by vocational 
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CPHHICqLOM-BASED VOCATIOWAL ASSgSSMgWT HATING FORM 



COSMETOLOGY 



COUHSB LEHGTH: 9 VCS 18 VU 



M P 



gnJDENT 



LAST NAME 



SEI QHADE SCHOOL 



0IHBCT10M5: 1. DO HOT RATE AKT BEHAVIOR OR OtJTCOKE MOT TAOGHT OR OBSERVED. 

2. CHECK P IF THB STUDEHT CAB PERFORM AT THE IHSTRUCTOR'S LEVEL OP 
EXPECTATION. 

3. CHECK M IF THE STTOBBT CAB PERFORM AT THE IBSTRUCTOM'S LEVEL OF 
nPECTATIOB WHEB IBSTRaCIION HAS BEEN MODIFIED. 

H. UST ADDITIONAL BEHAVIORS AND OOTCOMES AS NEEDED. 



L WORK RELATED BEHAVIORS 



III. OCCUPATIONAL SKILL OUTCOMES 
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? . GiTa a razor out 



8. Larar'^cut hair 



9. Nalata with taacfaara/aupanrlaora 



10. 

lit. 

22±. 
u. 

lii 

21^ 

Hi 
2£i 
21. 



Cooparata aa a taa« aaabar 



9. Slither hair 
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GENERAUZED EMPLOYABILITY OUTCOMES 
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teachers, of 174 instructor respondents in the pilot study, 94,8 percent 
indicated that the work related behavior items were afpropriate for xheir 
courses. The generalized enployability outccroes were considered appropriate 
by 85.5 percent of the respondents and 82.7 percent found the specific occu-- 
pational skill outccanes to be appropriate. No major changes were rccommended 
for work related behaviors or generalized enployability outcomes f o] lowing the 
fomative evaluation. Several of the ooci?>ational skill outcomes were modified 
based on reccramendatioas fran the content area teachers. 

Initial tests of CBVA instrument reliability in the 1986 fomative evaluation 
were cQso encouraging. A coefficient of ocaigruenoe was calculated for the 
21 work related behaviors and 16 enployability outocmes variables for paired 
teacher raters using 171 student observations after 9 weeks of instruction. 
The procedure provided a measure of pattern as well as magnitude similarity. 
The coefficient of congruence for work related behaviors ranged from 0.867 to 
l.OOO, with a mean coefficient of 0.946. For generalized enployability out- 
ccanes, the coefficient ranged from 0.924 tc 1.000, with a mean coefficient of 
0.969. The coefficients indicated effective inter-rater agreement. 

Proficiency levels of students were determined by ccirputing percentages of 
teachers' ratings for each Listructor in 7 content areas at middle/junior 
high school g.ade levels and 11 content areas offered in hic^ schools. 

TWO levels of proficiency could be rated. An item rated as being proficient 
indicated that the student attained the proficiency independent of any 
significant teacher assistance. An item rated as proficient with modification 
indicated that the student attained proficiency with some type of additional 
assistance being required. Teachers alone decide v*at constitutes "pro- 
ficiency" or "additional assistance." The CBVA instrument was designed 
to only generate ratings for satisfactory performance. 

Proficiency ratings were then grouped according to 21 work related behaviors 
and 16 generalized enployability outccines standardized across all vocational 
courses and CWE. The specific occi^tional skill aitcomes were initially 
selected and validated by teachers for eadi vocation? 1 cjurse. The occupa- 
tional skill outcomes were represented by selections c? student learning 
objectives taken from the DoDDS curriculum manuals publisiied for ead^i content 
area. Teachers select those occupational skill outcoties -u'lat apply and add 
others from their programs, if necessary. The proficiency levels were de- 
termined by calculating the average number of occi?>ational skill outcomes 
attained in each content area from the total number identified. 

Correlations of ratings provided by hic^ school vocational teachers and 
those of jc*) si?Dervisors were made to determine how well the CBVA data 
furnished by hi^ school vocationsd teachers rlc^t be used to predict 
future student performance in work related beiiaviors and generalized 
enployability outocmes in real work environments. The peculation in- 
cluded 127 students enrolled in grades 10, 11, and 12 v^o were rated by 
their vocational teachers and 96 CWE students v^o were rated by their job 
sipervisors. The Pearson Product-Monent Correlation Coefficient was 
applied to test the relationships between the two sets of paired data. 
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Vocational teachers also rated individual students on interest shown for the 
course during the instructional period and on the level of aptitude students 
appeared to demonstrate for the content area. Ratings were e>^ressed as being 
either hic^, iredium, or low for eacii category. Average ratirgs for each group 
were converted to peroents. 

The amount of additional teacher assistance needed by students to become 
proficient was calcolated as a percentage of their total prof i.ciency ratings 
across all vocational course offerings for each item listed in the work re- 
lated behaviors and generalized employability cxxtccmes. The amoun- of teacher 
assistance provided was also determined on student ratings of occipational 
skill outcomes by subject area. 

Finally, the data were analyzed to observe differences between mcile and female 
students on all proficiencies gained and on the levels of interests and 
aptitudes reported. 

The results are based on data collected for 7 middle/junior high school voca- 
tional subjects and 11 senior high school vocational subjects in vtiich 
substantial numbers of students were enrolled. 

Charts I throu^ VI demonstrate student proficiency- levels in individual 
vocational subjects for the total number of outcomes rated in (1) v;ork related 
behaviors, (2) generalized employability outcomes, and (3) occupational skill 
outcoiies for grades 6 throu^ 8 and for grades 9 through 12. Middle and junior 
higli school students take the vocational courses for exploratory purposes. The 
length of courses may vary from 6 weeks to 18 weeks depending on individual 
school scheduling practices. Students in grades 9 through 12 were enrolled for 
a minimum of 18 weeks in every course. 

Work Related Behavior Ctoservations 

Middle/junior hic^ school students were found to express positive work re- 
lated behaviors ranging from 97 percent in the Business Enterprise course to 
69 percent in Educational Conputing, Chart I. By cteervation, there were no 
substantial differences in the performance of girls and boys. 

At the hic^ school level, Oiart II, the students performed from 100 percent 
in Automotive Technology to 61 percent in Graphics. Again, there were no 
substantial differences between male and female students except for Graphics 
v*iere girls were rated 26 peroent points lower than boys and Elrafting viiere 
beys rated 22 percent points lower than girls. 

The hi^ behavior ratings received by special education students ajpeared 
to demonstrate positive social adaptability v*ien receiving instruction from 
vocational teachers in the mainstream programs. 
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GROUPED WORK RELATED BEHAVIORS: 

GRADES 6-8 
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GROL-PED WORK RELATED BEILWTORS: 

GRADES 9-12 
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Generalized Etnployabilitv cxitcxane Applications 



The category of "generalized enployability cutcones" refers to the student's 
proficiency in applying related academic skills/ displaying safety and 
equipment maintenance precautions, understanding the roles of employees and 
sipervisors, and in conceptualizing the functions of a business enterprise. 
The content is more abstract and revecds a measure of cognitive applications 
to specific vocational instruction. Since teactiers were instructed to limit 
their ratings to those generalized outocroes that they teach or otherwise 
observe, the ratings were not as oarprehensive or consistent in denotirjg 
student proficiencies involving such outocroes as the "application of science 
concepts" or the "understanding of enplqyer/enployee relationships," for 
exanple, as they w*ire for demonstrating work related behaviors in v*iich 
teachers have ocmmon interests. 

Middle/ junior hic^ school students, Chart III, demonstrated overall proficien- 
cies in generalized errployability outocsnes ranging frcxn 79 percent in Business 
Enterprise to 26 percent in Educational Computing. Major differences in the 
ratings between boys and girls were not evident. Chart IV reflects senior high 
school students proficiencies in generalized enployability outcomes. Profi- 
ciency ratings ranged from a hic^ of 71 percent in Cosmetology and CWE to a low 
of 27 i^ercent in Educational Ccmputinq. Boys rated sliq^itly higher than girls 
in 9 of the 11 vocational subjects. 

Examination of the rating forms revealed a reluctance of teachers to rate 
several of the academic skills associated with the vocational content. This 
was especially evident in Educational Conputing and Typing viiere the direct 
api;;.iication of academic skills is not as apparent in the more regimented learn- 
ing tasks associated with both courses. Another area v*iich was not rated by 
most teachers dealt with eitployee/enplqyer relationships and in conceptualizing 
the function of businesses. The Business Enterprise and CWE courses provide 
concrete work environments v*iere such outcomes are more easily observed. More 
ratings were given in those two subjects. 

Occupational Sid.lls Gained 

"Occupational s)cill outcomes" refer to those most functional vocational content 
objectives that students are e^qDected to aoccarplish in each course. Sub-skills 
generally have to be learned for students to achieve proficiency in an occupa- 
tional outocmne. The CBVA model was designed to assess functional outcomes 
rather than sub-skills because it was felt that the assessment profile would 
be more useful in itaking further decisions about vocational course selection 
and CWE placements. Further, as students progress throu^ various types of 
vocational courses, the sub skills are either learned or reinforced as occupa- 
tional skill outcanes develcp. Tbe acquisition of an occipational outcome, 
therefore, implies that dependent prerequisite skills have been learned. 

Chart v reflects the proficiencies of middle/junior hic^ school students at 
the esqploratory stage in oocipational skill outocmes. They ranged from 27 
pjeroent in Drafting to 52 percent in Clothing and Woods e The average ratings 
for achievement of occupational skill outccmes for boyj was 41.7 percent and 
43.2 percent for girls. Since the occupational skill outcomes listed on the 

7 



'2 



Chart m 
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GENERALIZED EMPLOYABIUTY OUTCOiMES: 

GRADES 6-8 




BUS ENT CLOTHINC 



-I i- 

rOOOS COMPUTERS 



TYPING DRAFTINC 



irOODS 



SUBJECTS 



100 
90-- 
80- 



ChartlV 
GROUPED 

GENERALIZED EMPLOYABILITY OUTCOMES: 

GRADES 9-12 




H 1 1 < -, 

AUTO GRAPH COSMET CLOTH rOODS COMP TYPING DRAFT ¥OODS JROTC CTE 

SUBJECTS 

8 



ERIC 



13 



student rating forms also include advanced level outcomes^ the acquisition of 
C3ver 40 percent of the skills by entry level stucJents in courses ranging from 6 
to 18 weeks duration is considered iirpressive. 

At the hi^ school level, Chart VI, the percentages in occupat skill 
outcoroes ranged fron 27 percent in Graphics to 68 percent in Clouiiing. The 
average achievement in occip??tianal skill outcoroes for boys was 54 percent 
and 52.4 percent for girls, Althou^ all courses are 18 weeks long with 
opportunities to take advanced instruction, itost handicapped students are 
reflecting entry level adiievement. 

To summarize, the proficiencies gained by mildly handicapped students within 
the vocatiCTial curriculum mainstream at all secondary levels appear to be 
quite acceptable. Of premising note, are the exceptionally hic^ proficiency 
ratings in work related behaviors. Such ratings not only demonstrate an ad- 
mirable quality of socicil behavior on the part of handicapped students, but 
cQso demonstrate the value vocational instructors place on teaciiing behavioral 
objectives. Such performance helps lay to rest pernicious thinking that 
handicajped students will be behavioral problems in vocational classes. 

Another positive finding is the level of occipationcil skill development gained 
by middle/junior hi^ school students in short term ejq^loratory subjects for 
which prior vocational assessment information is not available. Uieir per- 
formance stpports the system's premise that handicapped stixJents need to be 
enrolled in various vocational subjects, regardless of background diagnostic 
information, to prevent possible exclusion from vocational instruction due to 
diagnostic bias that may result fron evzduating students v*io lack initial 
learning e^qDeriences. The CBVA assessment procedure reflects student perfor- 
mance as products of currenv educational esq^erience, leaving little room for 
diagnostic voids or unthou^tful interpretations. 

Student Interests and Aptitudes 

The CBVA instrument provides opportunities for vocational teachers to estimate 
each student's interest in the course as well as aptitude for the content. 

Chart VII, displays the results of interest and aptitude ratings for middle/ 
junior hi^ school students. It can be observed that student interests tend 
to run at average and hi^ average levels. Aptitude ratings for the explora- 
tory subject areas were similar thou^ from 5 to 18 percent Icwer than in- 
terest levels. Students displayed hi^est aptitude for the Business Enterprise 
course, 70 percent , and lowest for lypi^ ^ Drafting, 52 percent. 
Differences by gender were observed to be sli^t. 

At the hic^i school level, interest in vocationcil subjects generally fell in 
the hi^ average to hi^ range. Chart VIII. The courst commanding hi<^est 
interest were CWE, JROTC, Cothing, and Graphic Arts. ' .le lowest interest, 
65 percent, was found to be in Woods. Boys rated consi erably higher in 
Automotive Technology, Graphic Arts, and Vtoods; Girls in Cosmetology, 
Bducationed Conpiting, and Foods. 
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Chart V 

OCCUPATIONAL SKILLS OUTCOMES: GRADES 6-8 
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Chart VII 
INTEREST/ APTITUDE: GRADES 6-8 
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Chart Vlii 
INTEREST/APTITTJDE: GRADES 9-12 
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While hi<^ schcxjl student aptitudes for tlie subject areas were rated Icwer 
than interests, they all fell within the average to hi<^--average ranges. 
Girls received sli^tly hi^ier aptitude ratings than boys in 7 of the 11 
subjects. Ihe hi^iest aptitude rating, 70 percent, was in CWE vtiere the 
iteasure of success tends to be on what students can do in work environments 
rather than on vAiat th^ appear to knew from performance on classroom 
examiiiations. 

Work Related Behaviors and Generalized Enrplovability CXitcoroes Profiles 

When the 21 work related behaviors and the 16 generalized enployability 
cutoones are iiidividually examined across the spectrum of vocational 
subjects, the ratings yield information viiich can be used to better 
understand oaranon strengths and limitations of student performance as 
well as possible instructional gaps to be oQnsidered for additional 
program emphasis* 

Students in grades 6-8 displayed a consistently hiq^i performance pcittem of 
work related behaviors. Chart IX. All work related behaviors were rated 
above 75 percent. Girls tended to rate sli^tly hi^er than boys. 

Hi^ school students. Chart X, also rated hi<^ on work related behaviors with 
the lowest rating being 78 percent. Differences between the performance of 
boys and girls were negligible. 

A diverse pattern of generalized enployability outcomes emerged for students 
at both secondary levels* In grades 6-8, Chart XI, students were rated profi- 
cient from 52 percent to 87.5 percent in their application of related academic 
slcills with the exception of "science conoqjts" viiidi was rated at 19 percent. 
Students also achieved proficiencies above 50 percent in "practicing related 
hygienic requirements," "demonstrating safety precautions," "demonstrating 
problen solving techniques," and "shewing interest in the occi^tional area." 
Teachers rated fewer students as being proficient in "performing equipment 
maintenance, " 32 percent, "understanding various etrployer/ettployee 
relationships," 21 percent to 29 percent, and in "understanding business 
enterprise functions," 17 percent. Ihe performance of middle/junior high 
school boys and girls was similar. 

Chart XII indicates the proficiencies acihieved in generalized enployability 
oitocBdies by senior hi^ school students. Althou^ the proficiency levels for 
"applying related science concepts" iiiproved, the outcaroe (27 percent) was 
still rated lowest for the grotp of related academic skills. Strengths 
included proficiencies in "understanding oral directions, (87 percent) and 
"understanding written directions" (79.5 percent) . Other strengths above 60 
percent included "applying related terminology, " "practicing related hygienic 
requireainents," and "demonstrating safety precautions." Ihe lowest enploya- 
bility outccroe (19 percent) was in "understanding the function of a business 
enterprise." 

It would appear that vocational teachers, vdiile demonstrating success in 
teaching student work related behaviors, enployability outcomes, and 
ooctpational skills, may not be emphasizing how those traits contribute to 
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Chart DC 

INDIVIDUAL WORK RELATED BEHAVIORS: 

GRADES 6-8 
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Chart X 

ESDIVIDUAL WORK RELATED BEHAVIORS: 

GRADES 9-12 
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Chart XI 

INDIVIDUAL GENERALIZED EMPLOYABIUTY 
OUTCOMES: GRADES 6-8 
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the success of a business enterprise, the source of nost occipatiorial oppor- 
tunities. In fact, the fewest proficiency ratings anong the generalized 
enplqyability outcanes were found to be those eissociated with understanding 
business related roles and conc^>ts. 

Teacher Assistance and Student Perfonnano e 

Chart XIII displays the average amount of additional assistance middle/ 
junior hic^ school teachers provided students. On the 21 work related 
bdiaviors, 33 percent of the total proficiency levels aciiieved by boys 
was attributed to extra teacher assistance. Girls required only 21 percent 
teacher assistance to obtain their tdtal proficiency levels. Teachers gave 
boys more help than girls. Uie extra help e^pears to pay off as total profi- 
ciencies are about equal between the sexes. Outocmes requiring the most 
teacher assistance by both sexes were **making judgments and decisions/* 
"corpleting tasks accurately," and "ocnpleting tasks in a timely manner." 
Those requiring the least were "displaying apprcpriate appearance," 
"ccnplying with attendance regulations," and "practicing punctuality." 



Chai t XIII 

TEACHER ASSISTANCE ON 

WORK RELATED BEHAVIORS: GRADES 6-8 




1. t. a. 4. 5. 7. a. 1. 10. 11. IS. la. u. le. it. it. it. to. ti. 

UEIIAVIOUS 



On the 16 generalized enplqyability outocroes, Chart XIV, middle/ junior high 
sdiool students benefitted most frtxn teacher assistance in outcxmes includ- 
ing "understanding oral directions/' "understanding writter* directions/' 
"allying related \:enninology/' "applying related measurement functions/' 
and "demonstrating prci)l€m solving tediniques." 
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Those requiring miniinal teacher assistance were outcomes vfriere students also 
demonstrated lew overall proficiency levels to include "understanding eirployer/ 
enployee roles," "understanding the function of a si?)ervisor, " and "under- 
standing the role of'' enployee evaluation." The average anount of extra 
assistance provided to boys was 25.6 percent and to girls was 22.2 percent of 
their respective total proficiency levels. Ihe question of v*iether students 
did not perform as well due to their ability or as a consequence of the content 
not being addressed in the instructional program, was not determined, but the 
fact that teachers did not provide the level of assistance that they did in 
academic areas suggests that ernplqyer/errployee relationships were not stressed 
in the vocational curriculums. 



Chart XIV 
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Data were also collected for each of the 18 vocational courses on occupational 
skill outcones. Middle/ junior hi^ school students we'^ given special assis- 
tance on 21 percent of the total proficiencies acquired, Chart XV. Differences 
occurring by vocational course content varied but can only be properly analyzed 
by eadi teacher since the outcomes criteria can be modified for any course by 
any teadier. 

For hi^ sdiool students, the average amount of assistance provided by teachers 
on work related b^viors is displayed on Chart XVI. Boys were assisted on 
16 percent and Girls on 28 percent of the total proficiencies gained. Ihe 
extra assistance brou^t the total performance of boys up to the levels 
achieved by the girls and both groi^ appeared to benefit significantly from 
such assistance. 
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Chart XY 



u 

aUTUlNC 



TEACHER ASSISTA>'CE ON 

OCCUPATINAL SKILLS OUTCOMES: 
GRADES 9- 12 




rOOOS COMPUTCIIS TYPINC 

SUUJECTS 



MUmMC 



CIURTXVI 

TEACHER ASSISTANCE ON 
WORK RELATED BEHAVIORS; 
GRADES 9-12 




7. t 



t. 10. 11. It. IS. 14. 16. 10. 17. It. It. to. tl. 

BEHAVIORS 



17 

?2 



students received nost help in "working unsupervised," "corrpleting tasks 
a<xaarately," and "oonpleting tasks in a tiinely manner." Less assistance 
was needed on "relating with peers" ard "displaying apprcpriate a^jearance." 
On generalized eraployability outooroes. Chart XVII, the average amount of 
assistance for hi^ school boys (24 percent) , and girls (20 percent) was 
considerably less as were the toted proficiencies gained. However, the 
amount of assistarx3e made important contributions to their overall achieve- 
ment levels. Students received most help in "understanding oral direc- 
ticais" and '•understanding written directions." Those outccannes vhere 
students required less assistance were "understanding the business enter- 
prise," and "applying related science ccno^Dts." As with the middle/ 
junior hi^ students, those outccroes vtfiere students received less 
assistajx^e were also those vhere students received fewer proficiency 
ratings. 

Chart XVIII for hi^ school students, indicates the average amount of 
assistance provided by teachers on occi5)ational skill development was 
23 percent of the total proficiencies achieved. Less assistance was 
provided in Graphic Arts and Educational Computing, and tl i most in 
Automotive Technology. 

It is clear that vocational teachers routinely make extra efforts in help- 
ing handicajped students achieve in their programs. It is also clear that 
the additional assistance provided makes important contributions to the 
overall performance of handicapped students. 

Predicting Transitional Job Success 

Ihe course offers the initial opportunity for handicapped students at 
grades 10, 11, and 12 to begin the transition from formal vocational train- 
ing and general education into the world of work. While the CBVA model was 
primarily designed to assist students and their parents to make individual 
educational decisions throu^out the middle/junior hic^ and high school 
years, the potential for using the data to predict jdD entry level success 
in coiiimunity work sites was also ej^lored. The ratings received by teachers 
for all 10th, 11th, and 12th grade students v*io oonpleted vocational courses 
were coanopared to the ratings given to all CWE students by their work site 
si5)ervisors for work related behaviors and generalized enployability outcomes; 
Chart XIX. 

Vocational teachers and work site sipervisors rated students as being profi- 
cient in work related behaviors at or above 84 percent. The correlation 
coefficient of .736 (significant at the .001 level) indicates that a posi- 
tive relationship exists between the ratings of teachers and those of job 
sipervisors. 

Ratings for generalized enployability outcomes were 51 percent from teachers 
and 69 percent frxxn work sipervisors. The correlation coefficient was .557, 
also significant at the .002 level. While both ratings would suggest that 
students have acquired the necessary behavioral and eraployability skills needed 
for entry level employment, work site si^jervisors rated student enployability 
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Chart XVII 
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outcomes 18 oentile points hi^er than the ratings received from vocational 
teachers. 



Farther examination of the generalized enployability outcomes, Chart XX, 
reveads that much hi^er ratings were received fran work supervisors in the 
areas of "understanding enployer/enployee roles," "supervision," "personnel 
evaluation," and "the function of a Business Enterprise." As discussed 
earlier, teachers may not rate these areas as often due to the nature of 
vocational instruction vAiich tends to einphasize ooccpational skill develop- 
ment in more fonnali;:ed classroom confines than xn actual work environments. 

It does appear that the ratings of vocational teachers do have predictive 
credibili^ when xjscd to assist students to enroll in CWE programs or when 
seeking entry level job placements upon graduation. 

POUDK-UP STODY OF CWE STUDENT RATINGS 

CWE is the system's primary course and activi* / v^iich contributes to the 
student transitioning process of applying basic and vocational education 
acoonplishments to the post-high school world of work. For this reason a 
re-check on the hi^ student ratings found in the i986-87 study by job site 
supervisors was conducted to verify the initial findings. 
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Ihe School Year 1987-88 data collection consisted of 191 CWE studp-nt ratings 
provided by 116 job supervisors. CWE students were rated as being proficient 
on 85 percent of the 21 work related behaviors. Seventy-five percent of the 
proficiencies gained were acccwplished without spe. ^ial assistance or modifi- 
cation of the assigiied tasks. The findings closely di?)licated the results 
of the initial study thereby adding credability to the Interpretation that 
handicapped students are demonstrating excellent social interaction and 
adaptability skills in ccxnmunity work envircannents. 

Students averaged 62 percent proficiency on the 16 generalized enployafcility 
cutocmes vitiidi r^resented a 7 percent drop in proficiency ratir»gs found in 
the intital study. Hcwever, the decrease in total job st?)ervisor ratings 
would provide a closer match to those of vocationcQ teachers (51 percent) 
vAiidi continue to be a significajit correllation between the r^.ters and also 
provides confidence in using vocational teacher CBVA ratings to predict 
student performaiice on real jobs. No major differences were found in the 
amount of Sfpecial cissistance job si:5)ervisors gave to Offi students. Over 
77 percent of OflE stiadents achievement was acquired without special assistance 
fron jd^ supervisors. 

Similau: successful patterns of achievement existed for students on occupational 
skills outcomes. Ihey were rated 73 percent proficient on th3 total number ot 
skills assigned and accomplished 77 percent of those skills without special 
assistance from job supervisors. 

Finally, a ccnparison of School Year 1987-88 student interest in and aptitude 
for CWF iob tasks were found to closely match the SY 1986-87 results. Interest 
ratings iirproved from 80 to 83 percent. ;^it\K3e ratings also inproved fror;. 
76 to 78 percent. Ihe Offi course clearly contributes to the career development 
transition needs of handicapped students based on the high interest, aptitude, 
and performance ratings consistently received from their job supervisors. The 
CBVA procedure provides hi^ily functional information about student perfor- 
mance, vAii^h is easily and routinely acq ired for each work placement experi- 
enced duriiig hi^ school. Such infonnation would appear to be inportant to 
decision-making activities that seek to locate entry level jobs for students 
ipon graduation. 

SUMMARY 

A CBVA model was developed, field tested, and implemented in 45 middle/junior 
hi^, and senior hi^ schools frcsn 1984 throu^ 198*^. A rating scale was 
designed for vocational teachers to use with mildly handicapped learning 
iirpaired students v*ien they were enrolled in vocational classes. A formatr e 
evaluation in 1986 provided feedback from 175 vocational teachers vrtiich also 
resulted in hi^ reliability and content validity measures on the rating scale. 
The program wets fully implemented in 36 middle/junior high schools and 28 high 
schools during School Year 1986-87. 

The demographic study was designed to discover the extent of student achieve- 
ment across 18 vocational courses in 21 work related behaviors, 16 generalized 
employability outcomes, and in 20-40 ocopational skills outcomes found in each 
vocational curriculum. Vocational teacher ratings indicated that students were 
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either proficient (independently) or proficient with extra teacher assistance. 
The grcx^p data provided a measure of systeirwide student achievenient in main- 
stream vocational courses, in work related social behaviors, in the aj^li ca- 
tion of academic skills to vocatia-al course ocxitent, and in the use of CBVA 
rating scales to predict expected student performance in transitional work 
environments. 

Specicd education students were rated in the upper quartile for all 21 work 
related behaviors in both middle/junior hi^ school and senior hi(^ school 
populations indicating positive social ad243tability v*ien receiving instruction 
in regular vocational classes. Performance on generalized employability skills 
was mixed. Students tended to rate average or better in a^^lying academic 
skills to vocational instruction and to be less proficient in categories re- 
lated to enployer-enplqyee relationships. An exception was in CWE where job 
supervisors rated students above average in the enplqyee-enployer rel *"^onship 
categories. 

Vocational teachers indicated that extra assistance was routinely provided 
and that the proficiency levels achieved by students were dramatically 
improved by such assistance. 

When vocational teacher ratings were correlated with those of CWE jcto 
supervisors for work related behaviors and generalized employability skills, 
the coefficients were significant for both categories, suggesting good 
potential for using the rating scales to predict performance in Ottl and other 
transitional job placements. 

The CBVA procedure generates corrprehensive information about individuals 
or groups which is easily acquired, inexpensive, and functionally useful for 
diagnostic purposes. 
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